00100	CAN EXPERT  JUDGES  ,  USING  TRANSCRIPTS  OF  TELETYPED  PSYCHIATRIC
00200	INTERVIEWS, DISTINGUISH HUMAN PARANOID PATIENTS FROM A COMPUTER MODEL
00300	OF PARANOID PROCESSES?                      COLBY AND HILF
00400	
00500	
00600	IN 1971 WE REPORTED THE CONSTRUCTION OF A CASE OF ARTIFICIAL PARANOIA
00700	IN THE FORM OF A COMPUTER SIMULATION [ ].  THIS HYPOTHETICAL  PATIENT
00800	PRODUCES  INPUT-OUTPUT  BEHAVIOR  CHARACTERISTIC OF PARANOID DIALOGUE
00900	INTERACTIONS IN A PSYCHIATRIC INTERVIEW.
01000	
01100	TO SIMULATE PROCESSES ONE WRITES AN ALGORITHM WHICH , WHEN RUN  ON  A
01200	COMPUTER  ,PRODUCES  PHENOMENA CHARACTERISITIC OF THOSE PROCESSES.  A
01300	SIMULATION IS  SUCCESSFUL  WHEN  ITS  BEHAVIOR  IN  SOME  CONTEXT  IS
01400	INDISTINGUISHABLE  FROM  THE  PROCESSES IT IS INTENDED TO SIMULATE. A
01500	SUCCESSFUL SIMULATION IS  ACHIEVED  BY  POSTULATING  A  STRUCTURE  OF
01600	INFORMATION-PROCESSING  MECHANISMS CAPABLE OF GENERATING THE BEHAVIOR
01700	IN QUESTION.  BY DESCRIBING THE POSTULATED  STRUCTURE  WE  PROVIDE  A
01800	THEORETICAL EXPLANATION OF THE PROCESSES BEING SIMULATED.
01900	
02000	ALTHOUGH  WE  DESCRIBED  THE 1971 MODEL IN DETAIL IN [ ], TO GIVE THE
02100	READER SOME IDEA OF HOW THE MODEL WORKS, WE SHALL  SKETCH  ITS  MAJOR
02200	OPERATIONS.   THE  ALGORITHM  IS  WRITTEN  IN  MLISP,  A  HIGH  LEVEL
02300	PROGRAMMING LANGUAGE , AND RUNS INTERACTIVELY  ON  THE  DEC  PDP-6/10
02400	TIME-SHARED  SYSTEM  OF THE STANFORD ARTIFICIAL INTELLIGENCE PROJECT.
02500	TO CONDUCT AN INTERVIEW AN INTERVIEWER, SITTING AT A VIDEO DISPLAY OR TELETYPE,
02600	TYPES  IN  AN EXPRESSION IN ORDINARY ENGLISH AND THEN RECEIVES AN ORDINARY
02700	ENGLISH REPLY FROM THE MODEL. THE INTERVIEWER  RESPONDS  IN  URN  AND
02800	THUS  THE INTERVIEW PROCEEDS. THE INTERVIEWER IS FREE TO SAY ANYTHING
02900	HE PLEASES, THE ONLY LIMITATION BEING THAT HE MAY NOT INPUT MORE THAN
03000	ONE SENTENCE OR QUESTION AT A TIME.
03100	
03200	NATURAL LANGUAGE EXPRESSIONS MADE BY ONE PERSON TO ANOTHER CONSTITUTE
03300	PERFORMATIVE ACTIONS IN WHICH THE SENDER OF THE  MESSAGE  INTENDS  TO
03400	AFFECT  THE  INTERPRETER IN SOME WAY. THE INTENTION MAY BE EXPLICITLY
03500	STATED IN THE MESSAGE (E.G.`TELL ME SOMETHING ABOUT YOURSELF') OR  IT
03600	MAY  BE  IMPLICIT  IN  THE  CONTEXT  OF  THE  DIALOGUE.   IN  EVERDAY
03700	CONVERSATIONS AN INTERPRETER  IN  THE  NORMAL  INFORMATION-PROCESSING
03800	MODE  DOES  NOT  INTENSIVELY  SCAN  THE  INPUT LOOKING FOR MALEVOLENT
03900	INTENTIONS ON THE PART OF THE SENDER. IN OUR MODEL WE POSTULATE  THAT
04000	IN  THE  PARANOID  MODE  THE DETECTION OF MALEVOLENCE BECOMES A FIRST
04100	PRIORITY. THUS THE PARANOID MODEL, UPON RECEIVING A NATURAL  LANGUAGE
04200	EXPRESSION,  ATTEMPTS TO DETERMINE THE INTENTIONS OF THE INTERVIEWER.
04300	THE MODEL TESTS THE INPUT FOR THE PRESENCE OF VERBAL  PATTERNS  WHICH
04400	ARE  CLASSIFIED  AS  MALEVOLENT,  BENEVOLENT OR NEUTRAL. THE PATTERNS
04500	REPRESENT COMBINATIONS OF  FEATURES,  PARTICULAR  WORDS  AND  PHRASES
04600	WHICH   ARE   INTERPRETED  AS  THE  INTENDED  MEANING  OF  THE  INPUT
04700	EXPRESSION.  WE DEFINE MALEVOLENCE AS AN INTENTION  TO  CAUSE  MENTAL
04800	HARM  OR  TO  MAKE  A  PHYSICAL  THREAT. ` MENTAL HARM' IS DEFINED AS
04900	HUMILIATION OR SUBJUGATION AND ` PHYSICAL THREAT' DENOTES A DIRECT OR
05000	INDUCED  ATTACK.  EXPLICIT  INSULTS  (E.G. `YOU ARE MENTALLY ILL') OR
05100	IMPLICIT INSULTS (E.G.   `HOW IS YOUR SEXLIFE?') ARE  INTERPRETED  AS
05200	HUMILIATION.    SUBJUGATION IS INTERPRETED FROM EXPRESSIONS REFERRING
05300	TO CONSTRAINT (E.G. YOU BELONG IN A HOSPITAL'.) OR COERCIVE TREATMENT
05400	(E.G.    `YOU  MIGHT  BE  PUT  IN  ISOLATION').  PHYSICAL THREATS ARE
05500	RECOGNIZED IN EXPRESSIONS OF DIRECT ATTACK  (E.G.`WE  WILL  GIVE  YOU
05600	ELECTRIC  SHOCK') OR OF INDUCED ATTACK (E.G. `DOES THE MAFIA KNOW YOU
05700	ARE HERE?').  BENEVOLENCE CONSISTS OF POSITIVE ATTITUDES TOWARDS  THE
05800	MODEL  AND  ITS  STORIES (E.G. `I WOULD LIKE TO HELP YOU', `I BELIEVE
05900	WHAT YOU SAY IS TRUE').   EXPRESSIONS WHICH  ARE  NOT  CLASSIFIED  AS
06000	MALEVOLENT  OR  BENEVOLENT ARE INTERPRETED AS NEUTRAL. IF NOTHING CAN
06100	BE RECOGNIZED IN THE INPUT, THE  MODEL  ASKS  A  QUESTION  ABOUT  THE
06200	INTERVIEWER'S  INTENTION,  (E.G.`WHAT  DO  YOU MEAN?'), CONTINUES THE
06300	TOPIC UNDER DISCUSSION, OR INTRODUCES A NEW TOPIC  DEPENDING  ON  ITS
06400	CURRENT STATE IN THE INTERVIEW AND WHAT HAS GONE ON BEFORE.
06500	
06600	AFTER  THE PATTERN FOUND IN THE INPUT EXPRESSION HAS BEEN CLASSIFIED,
06700	THE MODEL RESPONDS INTERNALLY AND EXTERNALLY. THE INTERNAL  RESPONSES
06800	CONSIST  OF ADJUSTING THREE AFFECT-VARIABLES LABELLED FEAR, ANGER AND
06900	MISTRUST. FOR EXAMPLE, IF MALEVOLENCE IS DETECTED IN THE INPUT, ANGER
07000	WOULD BE RAISED IN RESPONSE TO MENTAL HARM WHILE FEAR WOULD BE RAISED
07100	IN RESPONSE TO PHYSICAL THREAT. SINCE THE VARIABLE OF MISTRUST  IS  A
07200	FUNCTION  OF  FEAR  AND  ANGER  COMBINED,  IT  TOO WOULD RISE IN THIS
07300	INSTANCE.  WHEN THE INPUT EXPRESSION IS INTERPRETED AS BENEVOLENT  OR
07400	NEUTRAL,  THE AFFECT-VARIABLES DROP IN INTENSITY. THE EXTERNAL OUTPUT
07500	RESPONSE IN NATURAL LANGUAGE DEPENDS ON THE INTERPRETED NATURE OF THE
07600	INPUT,  THE TOPIC UNDER DISCUSSION (LOCAL CONTEXT), TOPICS PREVIOUSLY
07700	DISCUSSED   (GLOBAL   CONTEXT)   AND   THE   LEVEL   OF   THE   THREE
07800	AFFECT-VARIABLES.   THE  OUTPUT  STRATEGIES  ARE  INTENDED  TO REDUCE
07900	MALEVOLENT  ACTIONS  BY  RETRIBUTION   OR   WITHDRAWAL,   TO   PROMPT
08000	BENEVOLENCE  AND  TO  REPLY  FACTUALLY  TO  NEUTRAL INPUT. THE OUTPUT
08100	EXPRESSIONS ARE NOT GENERATED  WORD-BY-WORD  BUT  ARE  SELECTED  FROM
08200	LISTS  OF  PREFORMATTED  EXPRESSIONS. FOR EXAMPLE, IF MENTAL HARM HAD
08300	BEEN DETECTED IN THE INPUT, AND THE LEVEL OF ANGER IS HIGH WHILE FEAR
08400	AND  MISTRUST  ARE  MODERATE, THE OUTPUT EXPRESSION WOULD BE SELECTED
08500	FROM A LIST  OF  EXPRESSIONS  CLASSIFIED  AS  HOSTILE  COUNTER-ATTACK
08600	(E.G.`DO YOU KNOW WHAT YOU ARE DOING?').
08700	
08800	TWO  VERSIONS  OF  THE  MODEL,  WEAK  AND STRONG , CAN BE RUN. IN THE
08900	STRONG VERSION THERE EXISTS A DELUSIONAL COMPLEX ABOUT THE MAFIA  AND
09000	THE  VALUES  OF  THE  AFFECT-VARIABLES RISE MORE SHARPLY. IN THE WEAK
09100	VERSION  NO  FRANK   DELUSIONS   ARE   EXPRESSED,   ONLY   HINTS   OF
09200	SUSPICIOUSNESS  APPEAR,  AND  THE VALUES OF THE AFFECT-VARIABLES RISE
09300	MORE SLOWLY.
09400	
09500	THE MODEL DOES NOT ATTEMPT TO  ACCOUNT  FOR  HOW  PARANOID  PROCESSES
09600	DEVELOP.   IT  IS  LIMITED  TO  HOW THE PARANOID MODE OPERATES IN THE
09700	PRESENT. THE MODEL CHANGES DYNAMICALLY ONLY  OVER  THE  COURSE  OF  A
09800	SINGLE   INTERVIEW.    IN  EACH  SUCCEEDING  INTERVIEW  THE  STARTING
09900	CONDITIONS ARE THE SAME.
10000		BEING   RUDIMENTARY,   THE   1971   MODEL  HAS  A  NUMBER  OF
10100	DEFICIENCIES WHICH WE HOPE TO REMEDY  THROUGH  FURTHER  MODIFICATIONS
10200	AND   ADDITIONS.   IN   PARTICULAR,   THE  MODEL'S  NATURAL  LANGUAGE
10300	CAPABILITIES, BOTH IN UNDERSTANDING AND GENERATING EXPRESSIONS, NEEDS
10400	IMPROVEMENT.  ALSO A BETTER MODEL WOULD REQUIRE AN ABILITY TO EXAMINE
10500	AND REPORT ON ITS OWN STATES.TO EXTEND THE SCOPE OF THE SIMULATION  A
10600	RICH CONCEPTUAL MEMORY OF BELIEFS USEFUL IN MAKING INFERENCES IS ALSO
10700	NECESSARY.
10800	
10900	ONE METHOD FOR FINDING OUT WHETHER A SIMULATION IS SUCCESSFUL  IS  TO
11000	HAVE  EXPERT  JUDGES  CONDUCT  INDISTINGUISHABILITY TESTS.  IF EXPERT
11100	JUDGES,USING THEIR CONCEPTUAL MODEL  OF  THE  BEHAVIOR  IN  QUESTION,
11200	CANNOT  DISTINGUISH THE SIMULATION FROM ITS NATURAL COUNTERPART, THEN
11300	THE SIMULATION IS SUCCESSFUL, AT LEAST TO THE DEGREE MEASURED BY  THE
11400	INDISTINGUISHABILITY  TESTS  UTILIZED.  BUT  WHO IS AN EXPERT JUDGE?,
11500	WHAT DIMENSIONS OR PROPERTIES DOES HE USE IN MAKING  JUDGEMENTS?  AND
11600	HOW DOES HE JUSTIFY HIS JUDGEMENTS?
11700	
11800	WE  MIGHT  BE  ABLE TO ESTABLISH WHO IS AN EXPERT JUDGE OF PARTICULAR
11900	HUMAN BEHAVIORS BY A TEST PROCEDURE INVOLVING  DISCRIMINATION  TESTS.
12000	HOWEVER IN THE DOMAIN OF PSYCHIATRY AND PSYCHOPATHOLOGY THERE ALREADY
12100	EXIST CERTIFIED EXPERTS SUCH AS PSYCHIATRISTS, SOME BEING MORE EXPERT
12200	THAN  OTHERS.  ONE  DIFFICULTY  HERE  IN  ESTABLISHING  EXPERTISE  IS
12210	THE RELIABILITY OF WHAT IS BEING JUDGED. THAT IS, CAN CONSENSUS ABOUT
12300	PATHOLOGICAL BEHAVIOR BE  ACHIEVED  ?   WE  KNOW  THAT  MANY  OF  THE
12400	DIAGNOSTIC  CATEGORIES USED IN PSYCHIATRY ARE UNRELIABLE IN THE SENSE
12500	THAT ONLY LOW LEVELS OF INTERJUDGE AGREEMENT  CAN  BE  REACHED.   ONE
12600	EXCEPTION IS THAT INVOLVING THE CATEGORY OF 'PARANOID'. WE HAVE SHOWN
12700	HAT WHEN THE DATA BEING JUDGED CONSISTS OF  TRANSCRIPTS  OF  INITIAL
12800	PSYCHIATRIC  INTERVIEWS IN WHICH PATIENT AND PSYCHIATRIST COMMUNICATE
12900	BY MEANS OF REMOTELY LOCATED  TELETYPES,  HIGH  LEVELS  OF  AGREEMENT
13000	CAN BE REACHED AMONG RANDOMLY SELECTED PSYCHIATRISTS. FOR EXAMPLE, 34
13010	OUT OF 36 PSYCHIATRIST JUDGES AGREED AS TO WHICH OF TWO PATIENTS WAS 
13020	MORE PARANOID. [ ]
13100	HIGH LEVEL OF AGREEMENT WAS REACHED USING THE SPARSE  INFORMATION  OF
13200	SHORT TRANSCRIPTS (SEE FIGURES 1 AND 2) WHICH LACK NON-VERBAL CUES.
13300	
13400	
13500	SUPPOSE, HOWEVER,  THE  PATIENT  BEING  INTERVIEWED  BY  TELETYPE  IS
13600	ACTUALLY   A   COMPUTER   SIMULATION  OF  A  PARANOID  PATIENT.    DO
13700	PSYCHIATRISTS WHO INTERVIEW THE MODEL AGREE ABOUT  ITS  PARANOIDNESS?
13800	DO  PSYCHIATRISTS  WHO  READ  TRANSCRIPTS  OF INTERVIEWS CONDUCTED BY
13900	OTHER PSYCHIATRISTS ALSO AGREE REGARDING  THE  MODEL'S  PARANOIDNESS?
14000	THE  ANSWER TO BOTH OF THESE QUESTIONS IS `YES'. [ ] 
14100		IN THESE STUDIES THE JUDGES WERE NOT INFORMED THAT  ONE  OF  THE  INTERVIEWEES  WAS  A
14200	COMPUTER  MODEL.  NOW  THE QUESTION ARISES: WHEN INFORMED THAT ONE OF
14300	TWO TELETYPED PSYCHIATRIC INTERVIEWS IS WITH A COMPUTER MODEL AND ONE
14400	WITH AN ACTUAL PATIENT, CAN A PSYCHIATRIST TELL WHICH IS WHICH?
14500	
14600	
14700			METHOD AND RESULTS
14800	
14900	EIGHT  EXPERIENCED  PSYCHIATRISTS  (EXCLUDING  OURSELVES)   CONDUCTED
15000	INTERVIEWS  OVER  TELETYPES. EACH PSYCHIATRIST INTERVIEWED A PARANOID
15100	PATIENT AND OUR COMPUTER SIMULATION OF PARANOIA.TO AVOID  THE  ASKING
15200	OF  IRRELEVANT QUESTIONS, THE INTERVIEWERS WERE NOT INFORMED THAT ONE
15300	OF THE INTERVIEWEES WAS A SIMULATION. TRANSRIPTS OF THESE INTERVIEWS,
15400	ONE WITH A PARANOID PATIENT AND ONE WITH OUR PARANOID MODEL (EACH ONE
15500	CONDUCTED BY THE SAME PSYCHIATRIST) WERE SENT TO 100 POTENTIAL JUDGES
15600	RANDOMLY SELECTED FROM THE AMERICAN PSYCHIATRIC ASSOCIATION DIRECTORY
15700	AND WHO WERE ALSO LISTED IN THE DIRECTORY OF MEDICAL  SPECIALISTS  AS
15800	BEING BOARD CERTIFIED IN PSYCHIATRY.
15900	
16000		THE  JUDGES WERE ASKED TO DECIDE WHICH INTERVIEW WAS WITH THE
16100	PATIENT AND  WHICH  WITH  THE  COMPUTER  MODEL.  FORTY  PSYCHIATRISTS
16200	RESPONDED  WITH  21  (52%) MAKING THE CORRECT IDENTIFICATION WHILE 19
16300	(48%) WERE INCORRECT. BASED ON THIS RANDOM SAMPLE OF 40 PSYCHIATRISTS
16400	WE  ARE  95%  CONFIDENT THAT BETWEEN 37% AND 68% OF ALL PSYCHIATRISTS
16500	COULD CORRECTLY MODEL FROM PATIENT.THIS IS CLOSE TO THE 50% WE  WOULD
16600	EXPECT BY CHANCE.*
16700	
16800		CRITICS  MIGHT  ARGUE  THAT  WHILE  PSYCHIATRISTS  ARE EXPERT
16900	JUDGES OF PATIENT BEHAVIOR,  THEY  WOULD  PROBABLY  NOT  BE  ABLE  TO
17000	IDENTIFY   A   COMPUTER   PROGRAM,  AN  UNFAMILIAR  ENTITY  IN  THEIR
17100	EXPERIENCE. ON THE OTHER HAND COMPUTER SCIENTISTS,AS EXPERT JUDGES OF
17200	COMPUTER  PROGRAMS,  SHOULD  BE  ABLE TO DISTINGUISH THE MODEL FROM A
17300	HUMAN PATIENT.
17400	
17500	
17600		TO  TEST  THIS  HYPOTHESIS WE SENT THE SAME MATCHED INTERVIEW
17700	PAIRS, ONE WITH THE MODEL AND ONE WITH A  PATIENT,  TO  100  RANDOMLY
17800	SELECTED  MEMBERS  OF  THE ASSOCIATION FOR COMPUTING MACHINERY (ACM).
17900	OF THE SIXTY-SEVEN WHO RESPONDED, 32 (48%) WERE RIGHT  AND  35  (52%)
18000	WRONG.  BASED  ON THIS RANDOM SAMPLE OF 67 COMPUTER SCIENTISTS WE ARE
18100	95% CONFIDENT THAT BETWEEN 36% AND 60%  OF  ALL  COMPUTER  SCIENTISTS
18200	COULD  CORRECTLY  DISTINGUISH  THE  SIMULATION  FROM  A REAL PATIENT.
18300	AGAIN, THE PERCENTAGE IS CLOSE TO THAT EXPECTED FROM CHANCE GUESSING.
18400	
18500			DISCUSSION
18600	
18700	AFTER STUDYING TRANSCRIPTS OF TELETYPED INTERVIEWS, THE TWO GROUPS OF
18800	EXPERT  JUDGES,  PSYCHIATRISTS  AND  COMPUTER  SCIENTISTS,  WERE BOTH
18900	UNABLE TO CORRECTLY DISTINGUISH  PATIENT  FROM  COMPUTER  SIMULATION.
19000	THIS  IS  SURPRISING SINCE EVEN THOUGH PSYCHIATRISTS ARE NOT FAMILIAR
19100	WITH COMPUTERS, THEY HAVE HAD CONSIDERABLE EXPERIENCE IN INTERVIEWING
19200	HUMAN  PATIENTS.  LIKEWISE  WHILE  COMPUTER  SCIENTISTS  ARE  USUALLY
19300	UNFAMILIAR WITH HOW PATIENTS BEHAVE IN PSYCHIATRIC  INTERVIEWS,  THEY
19400	ARE  ACQUAINTED  WITH  THE  LIMITATIONS  AND  ABILITIES OF COMPUTERS.
19500	PERHAPS THE IDEAL JUDGES WOULD BE  THOSE  WHO  ARE  EXPERTS  IN  BOTH
19600	PSYCHIATRY  AND  COMPUTER  SCIENCE.IN  THE  IDEAL TEST, THEY WOULD BE
19700	ALLOWED TO CONDUCT THE INTERVIEWS THEMSELVES.  OUR 1971  MODEL  WOULD
19800	NOT  PASS  SUCH A SUCH A STRINGENT TEST BECAUSE OF ITS LINGUISTIC AND
19900	CONCEPTUAL LIMITATIONS.
20000			CONCLUSION		
20100		EXPERT JUDGES, PSYCHIATRISTS AND COMPUTER  SCIENTISTS,  COULD
20200	NOT  DISTINGUISH  A  SIMULATION MODEL OF PARANOID LINGUISTIC BEHAVIOR
20300	FROM THAT OF AN ACTUAL PARANOID PATIENT. WE ARE  WORKING  ON  FURTHER
20400	IMPROVEMENTS  AND  ADDITIONS  TO  THE MODEL TO ENABLE IT TO PASS MORE
20500	STRINGENT TESTS. 
20600	
20700	
20800	REFERENCES (FIGURES ARE ON LISTINGS)